Dynamic Visual Motion Estimation from Subspace Constraints

The problem of estimating rigid motion from projections may be characterized using a nonlinear dynamical system, composed of the rigid motion constraint and the perspective map. The time derivative of the output of such a system, which is called the “motion field” and approximated by the “optical flow”, is bilinear in the motion parameters, and may be used to specify a subspace constraint on either the direction of translation or the inverse depth of the observed points. Estimating motion may then be formulated as an optimization task constrained on such a subspace [4]. We pose the optimization problem in a system theoretic framework as the the identification of a nonlinear implicit dynamical system with parameters on a differentiable manifold, and use techniques which pertain to nonlinear estimation and identification theory to perform the optimization task in a principled manner. The application of a general method presented in [12] results in a recursive and pseudo-optimal solution of the visual motion estimation problem, which has robustness properties far superior to other existing techniques we have implemented. Experiments on real and synthetic image sequences show very promising results in terms of robustness, accuracy and computational efficiency. 1. MOTION ESTIMATION FROM A DYNAMIC MODEL Let a scene be represented by a set of N feature points in 3D space moving rigidly with respect to the viewer; the “visual motion estimation” problem is defined by the rigidity constraint and the perspective projection equations. If Xi = [X; Y; 2iIT are the coordinates of the i th point and xi A [zi yiIT the corresponding projections, we may write Research funded by the California Institute of Technology, a scholarship from the University of Padova, a fellowship from the “A. Gini” Foundation, an AT&T Foundation Special Purpose grant, ONR grant N0014-93-1-0990, grant ASI-RS-103 from the Italian Space Agency and the NSF National Young Investigator Award (P.P.). This work is registered as CDS Technical Report CIT-CDS 94-006, California Institute of Technology, January 1994 revised February 1994. where ni represents an error in measuring the pcssition of the projection of the point i and A represents an i-leal perspective projection. Solving the visual motion problem consists of estimating the ego-motion V, R from all tl e visible points, i.e. reconstructing the input of the abovt: system from its measured output. We show that it is pcssible to invert the above system using a technique which ias been recently introduced in [ 121 for identifying nonlinem implicit systems with parameters on a topological manifoltl. The scheme is motivated by the work of Heeger and Jepson [4, 51 and may be considered as a recursive solution of their task using methods which pertain to the field of nonlinear estimation and identification theory. As a result, the minimization task which is the core of the subspace method for recovering rigid motion needs not to be performed by extensive search, as it is done in [4]. Instead, an Implicit Extended Kalman Filter (IEKF) [2, 7, 8, 121 is i i charge of estimating the motion parameters recursively a :cording t o nonlinear prediction error criteria (for an intrc bductory treatment of Prediction Error Methods (PEM) in a linear context, see for example [14]). As a result, our method exploits in a pseudo-optimal manner the informatiox coming from a long stream of images, making the scheme robust and computationally efficient. 2. MOTION RECONSTRUCTION VIA INVERSION CONSTRAINED ON SUBSPACES Consider the following expression of the first deri-rative of the output of the model (1)) which is referred to as the “motion field”: